Structure from motion for drone videos

ABSTRACT

Aspects of the subject disclosure may include, for example, a method comprising obtaining, by a processing system including a processor, first and second models for a structure of an object, based respectively on ground-level and aerial observations of the object. Model parameters are determined for a three-dimensional (3D) third model of the object based on the first and second models; the determining comprises a transfer learning procedure. Data representing observations of the object is captured at an airborne unmanned aircraft system (UAS) operating at an altitude between that of the ground-level observations and the aerial observations. The method also comprises dynamically adjusting the third model in accordance with the operating altitude of the UAS; updating the adjusted third model in accordance with the data; and determining a 3D representation of the structure of the object, based on the updated adjusted third model. Other embodiments are disclosed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/238,349, filed Jan. 2, 2019, which is a continuation of U.S.application Ser. No. 15/455,745, filed Mar. 10, 2017 (now U.S. Pat. No.10,192,111), which are incorporated herein by reference in theirentirety.

FIELD OF THE DISCLOSURE

The subject disclosure relates to a system and method for usingStructure from Motion (SfM) techniques for processing video datacaptured by drones (small unmanned aircraft).

BACKGROUND

Structure from Motion (SfM) techniques are used to establishcorrespondence between a predefined three-dimensional (3D) model of anobject and two-dimensional (2D) images of the object. Models typicallyare developed for use with images from ground-based video equipment(ground-plane video) or images from video equipment installed inaircraft (aerial-plane video). Drones, however, generally operate atintermediate heights.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to the accompanying drawings, which are notnecessarily drawn to scale, and wherein:

FIG. 1 schematically illustrates collecting and processing ground-planevideo, aerial-plane video, and drone video image data, in accordancewith embodiments of the disclosure;

FIG. 2 schematically illustrates a procedure for generating a 3D SfMmodel for use in processing drone video data, in accordance withembodiments of the disclosure;

FIG. 3 schematically illustrates a SfM model, obtained using theprocedure of FIG. 2, applicable over a range of heights corresponding toa drone's operating range;

FIG. 4 schematically illustrates comparing the SfM model of FIG. 3 withimages obtained by a drone, and updating the model in accordance withembodiments of the disclosure;

FIG. 5 depicts an illustrative embodiment of a method used in portionsof the system described in FIGS. 1-2;

FIG. 6 depicts an illustrative embodiment of a communication system thatprovide media services to video processing systems of FIGS. 1-2;

FIG. 7 depicts an illustrative embodiment of a web portal forinteracting with the communication system of FIG. 6;

FIG. 8 depicts an illustrative embodiment of a communication device; and

FIG. 9 is a diagrammatic representation of a machine in the form of acomputer system within which a set of instructions, when executed, maycause the machine to perform any one or more of the methods describedherein.

DETAILED DESCRIPTION

The subject disclosure describes, among other things, illustrativeembodiments for using Structure from Motion (SfM) techniques to generatea 3D representation of an object observed by a drone. Other embodimentsare described in the subject disclosure.

One or more aspects of the subject disclosure include a methodcomprising obtaining, by a processing system including a processor, afirst model and a second model for a structure of an object; the modelsare based respectively on a ground-level or near-ground-levelobservations of the object and aerial observations of the object. Themethod also comprises determining model parameters for athree-dimensional (3D) third model of the object based on the firstmodel and the second model; the determining comprises a transferlearning procedure using a manifold. The method further comprisesobtaining data representing a third plurality of observations of theobject; the data is captured at an airborne unmanned aircraft system(UAS) operating at an altitude greater than that of the ground-levelobservations and less than that of the aerial observations. The methodalso comprises dynamically adjusting the third model in accordance withthe operating altitude of the UAS; updating the adjusted third model inaccordance with the data; and determining a 3D representation of thestructure of the object, based on the updated adjusted third model.

One or more aspects of the subject disclosure include a devicecomprising a processing system including a processor, and a memory thatstores executable instructions; the instructions, when executed by theprocessing system, facilitate performance of operations. The operationscomprise obtaining a first model and a second model for a structure ofan object; the models are based respectively on a ground-level ornear-ground-level observations of the object and aerial observations ofthe object. The operations also comprise determining model parametersfor a three-dimensional (3D) third model of the object based on thefirst model and the second model; the determining comprises a transferlearning procedure using a manifold. The operations further compriseobtaining data representing a third plurality of observations of theobject; the data is captured at an airborne unmanned aircraft system(UAS) operating at an altitude greater than that of the ground-levelobservations and less than that of the aerial observations; the datacomprises two-dimensional (2D) video images of the object. Theoperations also comprise determining correspondence parametersrepresenting a correspondence between the data and the third model;updating the correspondence parameters in accordance with the data;dynamically adjusting the third model in accordance with the operatingaltitude of the UAS; updating the adjusted third model in accordancewith the data; and determining a 3D representation of the structure ofthe object, based on the updated adjusted third model.

One or more aspects of the subject disclosure include a machine-readablestorage medium comprising executable instructions that, when executed bya processing system including a processor, facilitate performance ofoperations. The operations comprise obtaining a first model and a secondmodel for a structure of an object; the first model is based on a firstplurality of observations comprising ground-level or near-ground-levelobservations of the object, and the second model is based on a secondplurality of observations comprising aerial observations of the object.The operations also comprise determining model parameters for athree-dimensional (3D) third model of the object based on the firstmodel and the second model; the determining comprises a transferlearning procedure. The operations further comprise obtaining datarepresenting a third plurality of observations of the object; the datais captured at an airborne unmanned aircraft system (UAS) operating atan altitude greater than that of the first plurality of observations andless than that of the second plurality of observations. The operationsalso comprise dynamically adjusting the third model in accordance withthe operating altitude of the UAS; updating the adjusted third model inaccordance with the data; and determining a 3D representation of thestructure of the object, based on the updated adjusted third model andthe operating altitude.

FIG. 1 schematically illustrates a system 100 in which observations(video data) are collected relating to an object of interest (e.g.building 140). A drone 101, an aircraft 103, and a ground-based operator105 can acquire images of the building using video equipment 102, 104,106 respectively. As shown in FIG. 1, drone 101 operates above groundlevel in an operating range R (in this embodiment, a range of about 100feet to 300 feet above ground). Aircraft 103 typically operates at aheight of about 1000 feet or higher. The image data 112, 114, 116collected at the different heights will generally present differentperspectives and different levels of detail.

When comparing ground-plane, aerial-plane and drone video images of thesame object, it will be appreciated that drone videos typically havemore variation than aerial-plane videos due to the drone's loweroperating height, and can include more environmental effects thanground-plane videos (e.g. due to air drift and different weatherphenomena at different heights).

In an embodiment, the video observations (2D images) 112 from theequipment 102 of drone 101 are monitored and compared with apre-constructed 3D model of the scene being observed by the drone (inthis embodiment, building 140). The 3D model is derived from modelparameters for the scene based on the ground-plane observations 116 fromequipment 106 of ground-based operator 105, and the aerial-planeobservations 114 from the equipment 104 of aircraft 103. In anembodiment, the model parameters are generated using a transfer learningtechnique, as detailed below.

A machine-learning algorithm, executing on processing system 150, canthen be used to obtain SfM model parameters applicable to the drone'soperating range. Correspondence parameters (relating to the differencesbetween the 3D model and the 2D observations 112 from the drone) areadjusted to minimize the error between the model and the observations,so that a 3D representation of the object observed by the drone can begenerated.

FIG. 2 schematically illustrates a procedure 200 for generating a 3D SfMmodel applicable to drone video observations, using the system ofFIG. 1. In this embodiment, the image data 114 from the aircraft-borneequipment 104 is used to generate aerial-plane SfM model parameters 214,and the image data 116 from the ground-based equipment 106 is used togenerate ground-plane SfM model parameters 216. At a given height aboveground, the optimal 3D model (the best model for use by the drone atthat height) can be derived from the SfM model parameters for thatheight. However, those parameters (and thus the model itself) vary withheight in a non-linear fashion.

In this embodiment, a transfer learning approach is used on aGrassmannian manifold to determine the height-dependent 3D modelparameters. As is understood in the art, a Grassmannian manifold is aparticular category of a general non-Euclidean Riemannian manifold; theGrassmannian manifold operates on subspaces of data representations. Inthis embodiment, the subspace spanned by the SfM 3D model parameters forthe ground plane define one endpoint of a non-linear path on theGrassmannian manifold, and the subspace spanned by the SfM 3D modelparameters for the aerial plane define the other endpoint.

In this embodiment, transfer learning algorithm 240 identifies anoptimal non-linear path between the endpoints, subject to variousconstraints (e.g. a continuous and consistent path, physical boundariesof the 3D area of interest, a curved earth surface, etc.). Points alongthis path can then be sampled, corresponding to various heights aboveground, to determine optimal SfM parameters 250 for those heights. A 3Dmodel of the object is thus available for comparison with the 2D videodata 112 collected by the drone at a given height.

In an embodiment, system 150 continuously monitors the operating heightof the drone, and dynamically adjusts the 3D model to match that height.This permits real-time comparison of the model with images transmittedfrom the drone.

FIG. 3 is a schematic illustration 300 of a non-linear path 310 on aGrassmannian manifold, in accordance with embodiments of the disclosure.The subspaces spanned by the ground-plane and aerial-plane SfM modelparameters form the respective endpoints 311, 312 of the path. Points315 on the path represent optimal rendering of SfM parameters for aparticular height above ground. In an embodiment, path 310 is determinedand is then incrementally sampled over the operating range R of thedrone.

FIG. 4 is a schematic illustration 400 of active learning applied to theGrassmannian manifold of FIG. 3, in accordance with embodiments of thedisclosure. In an embodiment, live information (a real-time 2D image) issent by the drone and compared with SfM model parameters for the drone'soperating height. The model is then recalculated to correspond with thelive information. The updated model is represented by modified path 410in FIG. 4. Points 415 along the modified path can then be sampled todetermine updated SfM parameters for various heights.

In an embodiment, correspondence parameters are used to represent acorrespondence between the SfM 3D model and the actual observations fromthe drone. As more image data is obtained from the drone, thecorrespondence parameters can be adjusted so that error between thedrone's observations and the SfM 3D model is minimized.

In another embodiment, the path representing the 3D model can serve as abasis for estimates of a model for greater and/or lesser heights. Inthis embodiment, processing system 150 can apply an estimating procedureto extend the path beyond one or both of the endpoints. FIG. 4 showsmodified path 415 with extension 420 representing an estimated model. Ina further embodiment, if additional data is obtained for a heightoutside range R (e.g. represented by point 425), the optimal path can bere-calculated with a new endpoint.

FIG. 5 is a flowchart depicting a method 300 used in portions of thesystems described in FIGS. 1-2, in accordance with embodiments of thedisclosure. In steps 502-504, SfM 3D model parameters are determinedfrom ground-plane and aerial-plane observations respectively. A transferlearning procedure (step 506) is then used to build a 3D model,applicable to the drone's operating range, for the object of interest.In general, this 3D model is based on height-dependent SfM parametersvarying in a non-linear fashion. Accordingly, in an embodiment,variation of the model with height is represented as a non-linear pathon a Grassmannian manifold.

In step 508, the drone collects live data in the form of 2D images. Inan embodiment, the model is dynamically scaled in accordance with theoperating height of the drone. An active learning procedure (step 510)is then employed, in which the SfM 3D model parameters are updated inaccordance with the image data obtained by the drone. In an embodiment,this updating is performed in real time on the dynamically scaled model.In step 512, a SfM solution is calculated based on the updated modelparameters, to generate a 3D representation of the object observed bythe drone.

In an embodiment, the drone is in communication with processing system150 and has speed, altitude and direction sensors that transmitoperating data to the processing system. The processing system thus canpredict a new operating altitude of the UAS based on the operating data,and dynamically generate a predicted 3D representation of the structureof the object in accordance with the new operating altitude.

While for purposes of simplicity of explanation, the respectiveprocesses are shown and described as a series of blocks in FIG. 5, it isto be understood and appreciated that the claimed subject matter is notlimited by the order of the blocks, as some blocks may occur indifferent orders and/or concurrently with other blocks from what isdepicted and described herein. Moreover, not all illustrated blocks maybe required to implement the methods described herein.

FIG. 6 depicts an illustrative embodiment of a communication system 600for providing various communication services, such as delivering mediacontent. The communication system 600 can represent an interactive medianetwork, such as an interactive television system (e.g., an InternetProtocol Television (IPTV) media system). Communication system 600 canbe overlaid or operably coupled with the systems of FIGS. 1-2 as anotherrepresentative embodiment of communication system 600. For instance, oneor more devices illustrated in the communication system 600 of FIG. 6can include a device including a processing system that includes aprocessor, and a memory that stores executable instructions that, whenexecuted by the processing system, facilitate performance of operations.The operations can include obtaining a first model and a second modelfor a structure of an object; the models can be based respectively on aground-level or near-ground-level observations of the object and aerialobservations of the object. The operations can also include determiningmodel parameters for a three-dimensional (3D) third model of the objectbased on the first model and the second model; the determining caninclude a transfer learning procedure using a manifold. The operationscan further include obtaining data representing a third plurality ofobservations of the object; the data can be captured at an airborneunmanned aircraft system (UAS) operating at an altitude greater thanthat of the ground-level observations and less than that of the aerialobservations; the data can include two-dimensional (2D) video images ofthe object. The operations can also include determining correspondenceparameters representing a correspondence between the data and the thirdmodel; updating the correspondence parameters in accordance with thedata; dynamically adjusting the third model in accordance with theoperating altitude of the UAS; updating the adjusted third model inaccordance with the data; and determining a 3D representation of thestructure of the object, based on the updated adjusted third model.

In one or more embodiments, the communication system 600 can include asuper head-end office (SHO) 610 with at least one super headend officeserver (SHS) 611 which receives media content from satellite and/orterrestrial communication systems. In the present context, media contentcan represent, for example, audio content, moving image content such as2D or 3D videos, video games, virtual reality content, still imagecontent, and combinations thereof. The SHS server 611 can forwardpackets associated with the media content to one or more video head-endservers (VHS) 614 via a network of video head-end offices (VHO) 612according to a multicast communication protocol. The VHS 614 candistribute multimedia broadcast content via an access network 618 tocommercial and/or residential buildings 602 housing a gateway 604 (suchas a residential or commercial gateway).

The access network 618 can represent a group of digital subscriber lineaccess multiplexers (DSLAMs) located in a central office or a servicearea interface that provide broadband services over fiber optical linksor copper twisted pairs 619 to buildings 602. The gateway 604 can usecommunication technology to distribute broadcast signals to mediaprocessors 606 such as Set-Top Boxes (STBs) which in turn presentbroadcast channels to media devices 608 such as computers or televisionsets managed in some instances by a media controller 607 (such as aninfrared or RF remote controller).

The gateway 604, the media processors 606, and media devices 608 canutilize tethered communication technologies (such as coaxial, powerlineor phone line wiring) or can operate over a wireless access protocolsuch as Wireless Fidelity (WiFi), Bluetooth®, Zigbee®, or other presentor next generation local or personal area wireless network technologies.By way of these interfaces, unicast communications can also be invokedbetween the media processors 606 and subsystems of the IPTV media systemfor services such as video-on-demand (VoD), browsing an electronicprogramming guide (EPG), or other infrastructure services.

A satellite broadcast television system 629 can be used in the mediasystem of FIG. 6. The satellite broadcast television system can beoverlaid, operably coupled with, or replace the IPTV system as anotherrepresentative embodiment of communication system 600. In thisembodiment, signals transmitted by a satellite 615 that include mediacontent can be received by a satellite dish receiver 631 coupled to thebuilding 602. Modulated signals received by the satellite dish receiver631 can be transferred to the media processors 606 for demodulating,decoding, encoding, and/or distributing broadcast channels to the mediadevices 608. The media processors 606 can be equipped with a broadbandport to an Internet Service Provider (ISP) network 632 to enableinteractive services such as VoD and EPG as described above.

In yet another embodiment, an analog or digital cable broadcastdistribution system such as cable TV system 633 can be overlaid,operably coupled with, or replace the IPTV system and/or the satelliteTV system as another representative embodiment of communication system600. In this embodiment, the cable TV system 633 can also provideInternet, telephony, and interactive media services. System 600 enablesvarious types of interactive television and/or services including IPTV,cable and/or satellite.

The subject disclosure can apply to other present or next generationover-the-air and/or landline media content services system.

Some of the network elements of the IPTV media system can be coupled toone or more computing devices 630, a portion of which can operate as aweb server for providing web portal services over the ISP network 632 towireline media devices 608 or wireless communication devices 616.

Communication system 600 can also provide for all or a portion of thecomputing devices 630 to function as a media content server (hereinreferred to as server 630). The server 630 can use computing andcommunication technology to direct content to the initial and targetpresentation devices and transmit messages, which can include amongother things, transmitting the advance portion of the content, inaccordance with method 500. The media processors 606 and wirelesscommunication devices 616 can be provisioned with software functions toutilize the services of server 630. For instance, functions of mediaprocessors 606 and wireless communication devices 616 can be similar tothe functions described for the devices of FIGS. 1 and 2, in accordancewith method 500.

Multiple forms of media services can be offered to media devices overlandline technologies such as those described above. Additionally, mediaservices can be offered to media devices by way of a wireless accessbase station 617 operating according to common wireless access protocolssuch as Global System for Mobile or GSM, Code Division Multiple Accessor CDMA, Time Division Multiple Access or TDMA, Universal MobileTelecommunications or UMTS, World interoperability for Microwave orWiMAX, Software Defined Radio or SDR, Long Term Evolution or LTE, and soon. Other present and next generation wide area wireless access networktechnologies can be used in one or more embodiments of the subjectdisclosure.

FIG. 7 depicts an illustrative embodiment of a web portal 702 of acommunication system 700. Communication system 700 can be overlaid oroperably coupled with systems 100 and 200 of FIGS. 1 and/or 2, asanother representative embodiment of communication system 600. The webportal 702 can be used for managing services of systems 100 and 200 ofFIGS. 1 and/or 2, and communication system 600. A web page of the webportal 702 can be accessed by a Uniform Resource Locator (URL) with anInternet browser using an Internet-capable communication device such asthose described in FIGS. 1 and/or 2. The web portal 702 can beconfigured, for example, to access a media processor 406 and servicesmanaged thereby such as a Digital Video Recorder (DVR), a Video onDemand (VoD) catalog, an Electronic Programming Guide (EPG), or apersonal catalog (such as personal videos, pictures, audio recordings,etc.) stored at the media processor 406. The web portal 702 can also beused for provisioning Internet services, provisioning cellular phoneservices, and so on.

The web portal 702 can further be utilized to manage and provisionsoftware applications to adapt these applications as may be desired bysubscribers and/or service providers of systems 100-200 of FIGS. 1and/or 2, and communication system 600. Such applications can be used toextend the native sharing functions of the STB and mobile devicedescribed above. For instance, subscribers can log into their on-lineaccounts and provision the server 630 with contact information to enablecommunication with devices described in FIGS. 1 and 2. Service providerscan log onto an administrator account to provision, monitor and/ormaintain the systems 100-200 of FIGS. 1 and/or 2 or server 630.

FIG. 8 depicts an illustrative embodiment of a communication device 800.Communication device 800 can serve in whole or in part as anillustrative embodiment of the devices depicted in FIGS. 1-2 and FIG. 6,and can be configured to perform portions of method 500 of FIG. 5.

Communication device 800 can comprise a wireline and/or wirelesstransceiver 802 (herein transceiver 802), a user interface (UI) 804, apower supply 814, a location receiver 816, a motion sensor 818, anorientation sensor 820, and a controller 806 for managing operationsthereof. The transceiver 802 can support short-range or long-rangewireless access technologies such as Bluetooth®, ZigBee®, WiFi, DECT, orcellular communication technologies, just to mention a few (Bluetooth®and ZigBee® are trademarks registered by the Bluetooth® Special InterestGroup and the ZigBee® Alliance, respectively). Cellular technologies caninclude, for example, CDMA-1X, UMTS/HSDPA, GSM/GPRS, TDMA/EDGE, EV/DO,WiMAX, SDR, LTE, as well as other next generation wireless communicationtechnologies as they arise. The transceiver 802 can also be adapted tosupport circuit-switched wireline access technologies (such as PSTN),packet-switched wireline access technologies (such as TCP/IP, VoIP,etc.), and combinations thereof.

The UI 804 can include a depressible or touch-sensitive keypad 808 witha navigation mechanism such as a roller ball, a joystick, a mouse, or anavigation disk for manipulating operations of the communication device800. The keypad 808 can be an integral part of a housing assembly of thecommunication device 800 or an independent device operably coupledthereto by a tethered wireline interface (such as a USB cable) or awireless interface supporting for example Bluetooth®. The keypad 808 canrepresent a numeric keypad commonly used by phones, and/or a QWERTYkeypad with alphanumeric keys. The UI 804 can further include a display810 such as monochrome or color LCD (Liquid Crystal Display), OLED(Organic Light Emitting Diode) or other suitable display technology forconveying images to an end user of the communication device 800. In anembodiment where the display 810 is touch-sensitive, a portion or all ofthe keypad 808 can be presented by way of the display 810 withnavigation features.

The display 810 can use touch screen technology to also serve as a userinterface for detecting user input. As a touch screen display, thecommunication device 800 can be adapted to present a user interface withgraphical user interface (GUI) elements that can be selected by a userwith a touch of a finger. The touch screen display 810 can be equippedwith capacitive, resistive or other forms of sensing technology todetect how much surface area of a user's finger has been placed on aportion of the touch screen display. This sensing information can beused to control the manipulation of the GUI elements or other functionsof the user interface. The display 810 can be an integral part of thehousing assembly of the communication device 800 or an independentdevice communicatively coupled thereto by a tethered wireline interface(such as a cable) or a wireless interface.

The UI 804 can also include an audio system 812 that utilizes audiotechnology for conveying low volume audio (such as audio heard inproximity of a human ear) and high volume audio (such as speakerphonefor hands free operation). The audio system 812 can further include amicrophone for receiving audible signals of an end user. The audiosystem 812 can also be used for voice recognition applications. The UI804 can further include an image sensor 813 such as a charged coupleddevice (CCD) camera for capturing still or moving images.

The power supply 814 can utilize common power management technologiessuch as replaceable and rechargeable batteries, supply regulationtechnologies, and/or charging system technologies for supplying energyto the components of the communication device 800 to facilitatelong-range or short-range portable applications. Alternatively, or incombination, the charging system can utilize external power sources suchas DC power supplied over a physical interface such as a USB port orother suitable tethering technologies.

The location receiver 816 can utilize location technology such as aglobal positioning system (GPS) receiver capable of assisted GPS foridentifying a location of the communication device 800 based on signalsgenerated by a constellation of GPS satellites, which can be used forfacilitating location services such as navigation. The motion sensor 818can utilize motion sensing technology such as an accelerometer, agyroscope, or other suitable motion sensing technology to detect motionof the communication device 800 in three-dimensional space. Theorientation sensor 820 can utilize orientation sensing technology suchas a magnetometer to detect the orientation of the communication device800 (north, south, west, and east, as well as combined orientations indegrees, minutes, or other suitable orientation metrics).

The communication device 800 can use the transceiver 802 to alsodetermine a proximity to a cellular, WiFi, Bluetooth®, or other wirelessaccess points by sensing techniques such as utilizing a received signalstrength indicator (RSSI) and/or signal time of arrival (TOA) or time offlight (TOF) measurements. The controller 806 can utilize computingtechnologies such as a microprocessor, a digital signal processor (DSP),programmable gate arrays, application specific integrated circuits,and/or a video processor with associated storage memory such as Flash,ROM, RAM, SRAM, DRAM or other storage technologies for executingcomputer instructions, controlling, and processing data supplied by theaforementioned components of the communication device 800.

Other components not shown in FIG. 8 can be used in one or moreembodiments of the subject disclosure. For instance, the communicationdevice 800 can include a reset button (not shown). The reset button canbe used to reset the controller 806 of the communication device 800. Inyet another embodiment, the communication device 800 can also include afactory default setting button positioned, for example, below a smallhole in a housing assembly of the communication device 800 to force thecommunication device 800 to re-establish factory settings. In thisembodiment, a user can use a protruding object such as a pen or paperclip tip to reach into the hole and depress the default setting button.The communication device 800 can also include a slot for adding orremoving an identity module such as a Subscriber Identity Module (SIM)card. SIM cards can be used for identifying subscriber services,executing programs, storing subscriber data, and so forth.

The communication device 800 as described herein can operate with moreor less of the circuit components shown in FIG. 8. These variantembodiments can be used in one or more embodiments of the subjectdisclosure.

The communication device 800 can be adapted to perform the functions ofdevices of FIGS. 1 and/or 2, the media processor 606, the media devices608, or the portable communication devices 616 of FIG. 6. It will beappreciated that the communication device 800 can also represent otherdevices that can operate in systems of FIGS. 1 and/or 2, and incommunication system 600 such as a gaming console and a media player.

Upon reviewing the aforementioned embodiments, it would be evident to anartisan with ordinary skill in the art that said embodiments can bemodified, reduced, or enhanced without departing from the scope of theclaims described below. Other embodiments can be used in the subjectdisclosure.

It should be understood that devices described in the exemplaryembodiments can be in communication with each other via various wirelessand/or wired methodologies. The methodologies can be links that aredescribed as coupled, connected and so forth, which can includeunidirectional and/or bidirectional communication over wireless pathsand/or wired paths that utilize one or more of various protocols ormethodologies, where the coupling and/or connection can be direct (e.g.,no intervening processing device) and/or indirect (e.g., an intermediaryprocessing device such as a router).

FIG. 9 depicts an exemplary diagrammatic representation of a machine inthe form of a computer system 900 within which a set of instructions,when executed, may cause the machine to perform any one or more of themethods described above. One or more instances of the machine canoperate, for example, as the processing system 150 communicating withthe drone 101, the server 630, the media processor 606, and otherdevices of FIGS. 1-2 and FIG. 6. In some embodiments, the machine may beconnected (e.g., using a network 926) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient user machine in a server-client user network environment, or as apeer machine in a peer-to-peer (or distributed) network environment.

The machine may comprise a server computer, a client user computer, apersonal computer (PC), a tablet, a smart phone, a laptop computer, adesktop computer, a control system, a network router, switch or bridge,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. It will beunderstood that a communication device of the subject disclosureincludes broadly any electronic device that provides voice, video ordata communication. Further, while a single machine is illustrated, theterm “machine” shall also be taken to include any collection of machinesthat individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methods discussed herein.

The computer system 900 may include a processor (or controller) 902(e.g., a central processing unit (CPU)), a graphics processing unit(GPU, or both), a main memory 904 and a static memory 906, whichcommunicate with each other via a bus 908. The computer system 900 mayfurther include a display unit 910 (e.g., a liquid crystal display(LCD), a flat panel, or a solid state display). The computer system 900may include an input device 912 (e.g., a keyboard), a cursor controldevice 914 (e.g., a mouse), a disk drive unit 916, a signal generationdevice 918 (e.g., a speaker or remote control) and a network interfacedevice 920. In distributed environments, the embodiments described inthe subject disclosure can be adapted to utilize multiple display units910 controlled by two or more computer systems 900. In thisconfiguration, presentations described by the subject disclosure may inpart be shown in a first of the display units 910, while the remainingportion is presented in a second of the display units 910.

The disk drive unit 916 may include a tangible computer-readable storagemedium 922 on which is stored one or more sets of instructions (e.g.,software 924) embodying any one or more of the methods or functionsdescribed herein, including those methods illustrated above. Theinstructions 924 may also reside, completely or at least partially,within the main memory 904, the static memory 906, and/or within theprocessor 902 during execution thereof by the computer system 900. Themain memory 904 and the processor 902 also may constitute tangiblecomputer-readable storage media.

Dedicated hardware implementations including, but not limited to,application specific integrated circuits, programmable logic arrays andother hardware devices can likewise be constructed to implement themethods described herein. Application specific integrated circuits andprogrammable logic array can use downloadable instructions for executingstate machines and/or circuit configurations to implement embodiments ofthe subject disclosure. Applications that may include the apparatus andsystems of various embodiments broadly include a variety of electronicand computer systems. Some embodiments implement functions in two ormore specific interconnected hardware modules or devices with relatedcontrol and data signals communicated between and through the modules,or as portions of an application-specific integrated circuit. Thus, theexample system is applicable to software, firmware, and hardwareimplementations.

In accordance with various embodiments of the subject disclosure, theoperations or methods described herein are intended for operation assoftware programs or instructions running on or executed by a computerprocessor or other computing device, and which may include other formsof instructions manifested as a state machine implemented with logiccomponents in an application specific integrated circuit or fieldprogrammable gate array. Furthermore, software implementations (e.g.,software programs, instructions, etc.) including, but not limited to,distributed processing or component/object distributed processing,parallel processing, or virtual machine processing can also beconstructed to implement the methods described herein. Distributedprocessing environments can include multiple processors in a singlemachine, single processors in multiple machines, and/or multipleprocessors in multiple machines. It is further noted that a computingdevice such as a processor, a controller, a state machine or othersuitable device for executing instructions to perform operations ormethods may perform such operations directly or indirectly by way of oneor more intermediate devices directed by the computing device.

While the tangible computer-readable storage medium 922 is shown in anexample embodiment to be a single medium, the term “tangiblecomputer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “tangible computer-readable storage medium” shallalso be taken to include any non-transitory medium that is capable ofstoring or encoding a set of instructions for execution by the machineand that cause the machine to perform any one or more of the methods ofthe subject disclosure. The term “non-transitory” as in a non-transitorycomputer-readable storage includes without limitation memories, drives,devices and anything tangible but not a signal per se.

The term “tangible computer-readable storage medium” shall accordinglybe taken to include, but not be limited to: solid-state memories such asa memory card or other package that houses one or more read-only(non-volatile) memories, random access memories, or other re-writable(volatile) memories, a magneto-optical or optical medium such as a diskor tape, or other tangible media which can be used to store information.Accordingly, the disclosure is considered to include any one or more ofa tangible computer-readable storage medium, as listed herein andincluding art-recognized equivalents and successor media, in which thesoftware implementations herein are stored.

Although the present specification describes components and functionsimplemented in the embodiments with reference to particular standardsand protocols, the disclosure is not limited to such standards andprotocols. Each of the standards for Internet and other packet switchednetwork transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP) representexamples of the state of the art. Such standards are from time-to-timesuperseded by faster or more efficient equivalents having essentiallythe same functions. Wireless standards for device detection (e.g.,RFID), short-range communications (e.g., Bluetooth®, WiFi, Zigbee®), andlong-range communications (e.g., WiMAX, GSM, CDMA, LTE) can be used bycomputer system 900. In one or more embodiments, information regardinguse of services can be generated including services being accessed,media consumption history, user preferences, and so forth. Thisinformation can be obtained by various methods including user input,detecting types of communications (e.g., video content vs. audiocontent), analysis of content streams, and so forth. The generating,obtaining and/or monitoring of this information can be responsive to anauthorization provided by the user. In one or more embodiments, ananalysis of data can be subject to authorization from user(s) associatedwith the data, such as an opt-in, an opt-out, acknowledgementrequirements, notifications, selective authorization based on types ofdata, and so forth.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Theexemplary embodiments can include combinations of features and/or stepsfrom multiple embodiments. Other embodiments may be utilized and derivedtherefrom, such that structural and logical substitutions and changesmay be made without departing from the scope of this disclosure. Figuresare also merely representational and may not be drawn to scale. Certainproportions thereof may be exaggerated, while others may be minimized.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

Although specific embodiments have been illustrated and describedherein, it should be appreciated that any arrangement which achieves thesame or similar purpose may be substituted for the embodiments describedor shown by the subject disclosure. The subject disclosure is intendedto cover any and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, can be used in the subject disclosure.For instance, one or more features from one or more embodiments can becombined with one or more features of one or more other embodiments. Inone or more embodiments, features that are positively recited can alsobe negatively recited and excluded from the embodiment with or withoutreplacement by another structural and/or functional feature. The stepsor functions described with respect to the embodiments of the subjectdisclosure can be performed in any order. The steps or functionsdescribed with respect to the embodiments of the subject disclosure canbe performed alone or in combination with other steps or functions ofthe subject disclosure, as well as from other embodiments or from othersteps that have not been described in the subject disclosure. Further,more than or less than all of the features described with respect to anembodiment can also be utilized.

Less than all of the steps or functions described with respect to theexemplary processes or methods can also be performed in one or more ofthe exemplary embodiments. Further, the use of numerical terms todescribe a device, component, step or function, such as first, second,third, and so forth, is not intended to describe an order or functionunless expressly stated so. The use of the terms first, second, thirdand so forth, is generally to distinguish between devices, components,steps or functions unless expressly stated otherwise. Additionally, oneor more devices or components described with respect to the exemplaryembodiments can facilitate one or more functions, where the facilitating(e.g., facilitating access or facilitating establishing a connection)can include less than every step needed to perform the function or caninclude all of the steps needed to perform the function.

In one or more embodiments, a processor (which can include a controlleror circuit) has been described that performs various functions. Itshould be understood that the processor can be multiple processors,which can include distributed processors or parallel processors in asingle machine or multiple machines. The processor can be used insupporting a virtual processing environment. The virtual processingenvironment may support one or more virtual machines representingcomputers, servers, or other computing devices. In such virtualmachines, components such as microprocessors and storage devices may bevirtualized or logically represented. The processor can include a statemachine, application specific integrated circuit, and/or programmablegate array including a Field PGA. In one or more embodiments, when aprocessor executes instructions to perform “operations”, this caninclude the processor performing the operations directly and/orfacilitating, directing, or cooperating with another device or componentto perform the operations.

The Abstract of the Disclosure is provided with the understanding thatit will not be used to interpret or limit the scope or meaning of theclaims. In addition, in the foregoing Detailed Description, it can beseen that various features are grouped together in a single embodimentfor the purpose of streamlining the disclosure. This method ofdisclosure is not to be interpreted as reflecting an intention that theclaimed embodiments require more features than are expressly recited ineach claim. Rather, as the following claims reflect, inventive subjectmatter lies in less than all features of a single disclosed embodiment.Thus the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separately claimedsubject matter.

What is claimed is:
 1. A device comprising: a processing systemincluding a processor; and a memory that stores executable instructionsthat, when executed by the processing system, facilitate performance ofoperations comprising: obtaining data representing a plurality oftwo-dimensional (2D) video images of an object, the video imagescaptured using an airborne unmanned aircraft system (UAS) operating in afirst altitude range; constructing a three-dimensional (3D) model of theobject in accordance with the data, the constructing using ground levelor near-ground-level observations of the object obtained in a secondaltitude range and aerial observations of the object obtained in a thirdaltitude range, wherein the first altitude range comprises operatingaltitudes of the UAS above the second altitude range and below the thirdaltitude range; and dynamically adjusting the 3D model in accordancewith updated data representing a new video image captured at a newoperating altitude of the UAS.
 2. The device of claim 1, wherein theconstructing comprises a transfer learning procedure.
 3. The device ofclaim 2, wherein the 3D model is based on parameters varyingnon-linearly with an operating altitude of the UAS, and wherein thetransfer learning procedure is used on a manifold to determine theparameters.
 4. The device of claim 3, wherein the manifold is aGrassmannian manifold.
 5. The device of claim 3, wherein the parameterscomprise structure-from-motion (SfM) model parameters for the firstaltitude range.
 6. The device of claim 5, wherein the parameters aredetermined using a machine-learning algorithm.
 7. The device of claim 3,wherein variation of the 3D model with the operating altitudecorresponds to a non-linear path on the manifold.
 8. The device of claim7, wherein the non-linear path has a first endpoint corresponding to afirst model of the object based on a first plurality of observationscomprising the ground level or near-ground-level observations and asecond endpoint corresponding to a second model of the object based on asecond plurality of observations comprising the aerial observations. 9.The device of claim 1, wherein the operations further comprise:predicting, by the processing system, a future operating altitude of theUAS; and determining a predicted 3D model of the object in accordancewith the predicted future operating altitude.
 10. The device of claim 1,wherein the first altitude range is from about 100 feet to about 2000feet.
 11. A method comprising: obtaining, by a processing systemincluding a processor, data representing a plurality of two-dimensional(2D) video images of an object, the video images captured using anairborne unmanned aircraft system (UAS) operating in a first altituderange; constructing, by the processing system, a three-dimensional (3D)model of the object in accordance with the data, the constructing usinga first plurality of observations of the object obtained in a secondaltitude range and a second plurality of observations of the objectobtained in a third altitude range, wherein the first altitude rangecomprises operating altitudes of the UAS above the second altitude rangeand below the third altitude range; and dynamically adjusting the 3Dmodel in accordance with updated data representing a new video imagecaptured at a new operating altitude of the UAS.
 12. The method of claim11, wherein the constructing comprises a transfer learning procedure.13. The method of claim 12, wherein the 3D model is based on parametersvarying non-linearly with an operating altitude of the UAS, and whereinthe transfer learning procedure is used on a manifold to determine theparameters.
 14. The method of claim 13, wherein variation of the 3Dmodel with the operating altitude corresponds to a non-linear path onthe manifold.
 15. The method of claim 14, wherein the non-linear pathhas a first endpoint corresponding to a first model of the object basedon the first plurality of observations and a second endpointcorresponding to a second model of the object based on the secondplurality of observations.
 16. The method of claim 15, wherein the firstplurality of observations comprise ground level or near-ground-levelobservations of the object and the second plurality of observationscomprise aerial observations of the object.
 17. A non-transitorymachine-readable medium comprising executable instructions that, whenexecuted by a processing system including a processor, facilitateperformance of operations comprising: obtaining data representing aplurality of two-dimensional (2D) video images of an object, the videoimages captured using an airborne unmanned aircraft system (UAS)operating in a first altitude range; constructing a three-dimensional(3D) model of the object in accordance with the data, the constructingusing observations of the object obtained in an altitude range outsidethe first altitude range; and dynamically adjusting the 3D model inaccordance with updated data representing a new video image captured ata new operating altitude of the UAS.
 18. The non-transitorymachine-readable medium of claim 17, wherein the constructing the 3Dmodel of the object comprises a transfer learning procedure used on amanifold to determine parameters varying non-linearly with an operatingaltitude of the UAS.
 19. The non-transitory machine-readable medium ofclaim 18, wherein variation of the 3D model with the operating altitudecorresponds to a non-linear path on the manifold, wherein the non-linearpath has a first endpoint corresponding to a first model of the objectbased on ground level or near-ground-level observations of the objectand a second endpoint corresponding to a second model of the objectbased on aerial observations of the object.
 20. The non-transitorymachine-readable medium of claim 18, wherein the manifold is aGrassmannian manifold.